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1.  Objectives 

The  objective  of  the  current  grant  is  to  conduct  Unstructured  Solver  Performance 
Evaluation  activity  for  HPC  Institute  for  Advanced  Rotorcraft  Modeling  and  Simulation  (HI- 
ARMS).  Specified  deliverables  are  provided  to  the  HI -ARMS  evaluation  team  for  solutions  and 
computational  statistics  for  a  common  test  problem  on  a  NACA0015  wing,  and  optional 
deliverables  are  provided  for  solutions  on  a  realistic  helicopter  rotor,  which  will  be  used  as  the 
basis  for  measuring  the  attributes  of  the  current  unstructured  solver  to  meet  long-term  HI- ARMS 
objectives  and  program  requirements. 

2.  Computational  Approach 

The  tasks  under  the  current  grant  have  been  conducted  with  an  unstructured  flow  solver 
referred  to  as  U2NCLE  (Unstructured  Unsteady  Computation  of  Field  Equations).  It  is  a 
portable,  scalable  parallel  version  of  the  three-dimensional  unsteady  Reynolds-averaged  Navier- 
Stokes  flow  solver  developed  at  the  Computational  Simulation  and  Design  Center  (SimCenter)  at 
Mississippi  State  University.  The  essential  elements  of  this  capability  are  1)  a  dynamic  mixed- 
element  unstructured  grid  generation  procedure,  2)  a  portable  scalable  parallel  implicit  viscous 
flow  solver,  3)  a  6-DOF  trajectory  analysis  integrated  with  the  flow  solver  for  maneuvering 
predictions,  and  4)  a  visualization/animation  system  capable  of  processing  very  large  unsteady 
three-dimensional  datasets.  The  computational  simulation  methodology  used  has  evolved  over 
eighteen  years.  Detailed  accounts  of  the  unstructured  grid  generation  procedures  and  flow 
solvers  employed  here  are  given  in  previous  publications  [1-9]. 

The  present  report  will  focus  on  specific  topics  related  to  the  required  deliverables  as 
outlined  in  the  HI-ARMS  Invitation  to  Submit  for  Proposal  for  the  common  test  problem,  the 
NACA0015  wing,  and  computed  results  obtained  during  this  investigation  for  the  optional  test 
problem,  a  generic  but  realistic  three-bladed  rotor.  A  summary  of  capabilities  developed  and 
demonstrated  for  rotorcraft  related  simulations  is  given  here.  A  discussion  of  various 
computational  simulations  and  statistics  obtained  for  the  test  problems  then  follows. 

2.1  Multi-Element  Unstructured  Grid  Generation 

The  SimCenter’ s  unstructured  grid  generation  system  (SolidMesh)  was  used  to  generate  all 
of  the  grids  used  here.  SolidMesh  first  generates  surface  grids  for  all  surfaces,  and  then  uses  the 
Advancing  Front  Focal  Reconnection  scheme  (AFFR)  to  generate  high-quality  volume  grids. 
The  unstructured  grid  generation  schemes  used  here  are  given  in  [1-3].  Implicit  flow  simulation 
algorithms  are  used  to  enable  high  resolution  of  viscous  sublayer  regions  near  solid  boundaries. 
Tetrahedral  elements  are  not  well  suited  for  accuracy  in  viscous  regions.  Viscous  flow  accuracy 
can  be  improved  by  using  mixed  element  types.  A  multi-element  capability  has  been  developed 
for  this  purpose  and  used  in  the  wing  and  rotor  configurations.  The  mixed  element  capability 
utilizes  prisms  in  viscous  regions  near  the  solid  boundaries  and  tetrahedral  elements  in  inviscid 
regions,  with  pyramidal  elements  for  blending  between  these  regions.  Mixed  element  types  also 
provide  a  very  substantial  improvement  in  efficiency  and  run  time  because  the  number  of  edges 
is  greatly  reduced  below  that  required  for  tetrahedral  elements,  and  most  of  the  computational 
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labor  of  solution  is  contained  in  loops  over  edges.  Hexahedral  elements  are  also  supported  but 
were  not  used  here. 

Three  unstructured  meshes  were  generated  for  the  NACA0015  wing  configuration,  which 
include  1.57,  3.66,  and  7.0  million  node  points,  respectively.  The  first  mesh  (1.57M)  has  a 
coarser  surface  grid  resolution,  and  points  are  not  packed  near  the  wing  tip.  The  third  mesh 
(7.0M)  uses  fully  unstructured  surface  representation  for  the  wing  surface,  and  has  much  more 
refined  trailing  edge.  The  second  mesh  (3.66M)  has  a  similar  surface  point  distribution  as 
provided  by  HIARM  in  the  guideline  for  the  common  test  problem.  Therefore,  it  has  been 
selected  for  the  unstructured  solver  performance  evaluation,  see  Figure  1.  The  initial  spacing  off 
the  wing  surface  is  1.0  xlO"5,  which  results  in  an  average  y+  value  of  0.62  on  the  wing  surface  at 
the  free  stream  Mach  number  of  0.1235  and  a  Reynolds  number  of  1.5  million. 


Figure  1  Surface  meshes  for  the  common  test  problem: 
NACA0015  wing  with  3.66M  node  points 


2.2  Portable  Scalable  Parallel  Implicit  Solution  Algorithm 

U2NCLE  is  a  portable,  scalable  parallel  flow  solver  developed  for  use  with  multi-element 
unstructured  grid  topologies  that  provides  all  the  capabilities  needed  for  the  present  common  and 
optional  test  problems.  The  parallel  algorithms  used  here  [4-9]  have  evolved  over  the  past 
fifteen  years  from  previous  sequential  algorithms  for  the  unsteady  Reynolds-averaged  Navier- 
Stokes  equations.  The  parallel  flow  solver  employs  a  domain  decomposition  to  define  and 
distribute  work  among  multiple  processors,  and  to  utilize  distributed  memory  associated  with 
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multiple  processors.  The  solution  algorithm  uses  an  iterative  implicit  algorithm  at  each  time  step, 
applying  symmetric  Gauss-Seidel  relaxation  as  the  innermost  iteration,  to  provide  scalable 
concurrency.  Portability  across  numerous  parallel  computing  platforms  is  maintained  using  the 
MPI  message-passing  library  for  interprocessor  communications. 

There  are  three  versions  of  the  unstructured  solver  in  the  U2NCLE  simulation  package:  an 
incompressible  version  that  solves  artificial  compressibility  form  of  governing  equations,  a 
compressible  solver,  and  an  arbitrary  Mach  number  solver  that  uses  a  global  preconditioning 
technique.  A  thorough  discussion  of  the  incompressible  flow  solver  as  formulated  for  steady 
flow  problems  is  given  in  [4],  Discussions  of  the  arbitrary  Mach  number  flow  solver  as 
formulated  for  both  unstructured  and  structured  grids  are  given  in  [8,9].  Simulations  for  the  wing 
and  rotor  configurations  were  conducted  with  the  arbitrary  Mach  code  that  is  capable  of  solving 
both  low-  and  high-speed  flows.  A  simulation  and  prediction  capability  for  a  tiltrotor  that 
includes  the  actual  rotating  blades  and  an  icing  model  have  been  undertaken  to  develop  a 
physics-based  maneuvering  simulation  and  prediction  capability  for  complete  rotorcraft 
configurations  [10].  A  unique  feature  of  the  U2NCLE  solver  is  that  it  provides  a  physics-based 
capability  to  simulate  the  complex  flow  behavior  and  maneuvering  characteristics  for  complete 
tiltrotor  aircraft  that  includes  rotating  components,  such  as  the  rotating  blades  undergoing  cyclic 
blade-pitch  motion.  The  complex  geometries  are  addressed  using  dynamic  multi-element 
unstructured  grids  with  the  capability  of  moving  and  distorting  grid  motion  to  handle  moving 
appendages  and  rotating  blades.  The  numerical  algorithm  is  developed  based  on  a  node-centered 
finite-volume  implicit  scheme.  Newton  subiterations  are  used  to  advance  the  time-step,  and 
symmetric  Gauss-Seidel  relaxations  are  used  to  solve  the  linear  system  of  equations  at  each  time- 
step.  A  first-  or  second-order  temporal  accuracy  can  be  used  for  time-accurate  unsteady 
simulations.  An  inviscid  flux  is  calculated  using  a  modified  (preconditioned)  Roe  scheme,  and 
2nd-order  spatial  accuracy  is  achieved  by  using  a  least-squares  reconstruction  procedure.  Viscous 
effects  are  modeled  using  a  Spalart-Allmaras  turbulence  model,  or  k-  s ,  k-  co ,  and  q-  co  two- 
equation  turbulence  models. 

The  following  notable  capabilities  of  the  U  NCLE  solver  have  been  demonstrated  for 
advanced  rotorcraft  modeling  and  simulation: 

•  Solving  the  governing  equations  in  fixed  or  rotating  reference  frames  simultaneously. 

•  Rotor  blades  can  be  treated  as  actuator  disk  or  actual  rotating  blades. 

•  Realistic  blade  motion  including  cyclic  blade-pitch  motion. 

•  Loosely  coupling  with  elastic  blade  deformation. 

•  Realistic  rotating-blade/fuselage  clearances  (2%  and  3.4%  of  rotor  disk  diameter  were 
demonstrated). 

•  6-DOF  maneuvering  simulation  for  tiltrotor  aircraft  with  rotating  blades. 

•  Complete  rotorcraft  simulation  in  hover,  forward  flight,  and  various  descent  flight 
conditions. 

•  Coupling  for  two-dimensional  airfoils  with  an  icing  model  capable  of  treating  complex 
porous  ice  structures. 

•  Solving  both  low-  and  high-speed  viscous  flows  with  an  arbitrary  Mach  number  solution 
algorithm. 

•  Viscous  sublayer  resolution  such  that  y+  <  1  at  all  surface  points. 

•  Large  time  steps  corresponding  to  1-1.5  degrees  of  blade  rotation  per  step,  selection 
based  on  physics  and  accuracy  considerations  rather  than  numerical  stability  restrictions. 

•  Efficient  runtimes  and  parallel  efficiency  for  multiple  parallel  platforms  (Sun  El 0000, 
SGI  R12000,  Cray  T3E,  Linux  Cluster). 
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2.3  Visualization  and  Animation  Software 

The  visualization  and  especially  the  animation  of  complete  rotorcraft  simulations  has 
presented  a  significant  challenge  due  to  the  very  large  datasets  produced  by  time-accurate  three- 
dimensional  viscous  flow  simulations  with  rotating  blades,  using  grids  of  between  five  to  ten 
million  points.  Datasets  of  order  100  GB  are  typical,  and  visualization  capabilities  have  been 
developed  to  enable  such  animations  in  the  SimCenter’s  DIVA  visualization  software  [11]  to 
allow  processing  of  these  very  large  data  sets  efficiently.  All  viewgraphs  for  the  NACA0015 
wing  solutions  reported  here  were  produced  by  the  flow  visualization  software  DIVA,  and 
animations  of  the  three-bladed  rotor  flow  simulations  (see  Sec.  4)  that  include  high  frequency 
blade-pitch  oscillation  have  also  been  produced  with  DIVA.  The  following  Figure  2  shows  the 
iso-swirl  parameter  over  the  NACA0015  wing  visualized  by  DIVA,  indicating  the  strength  of  tip 
vortex  generated  in  the  flow  field.  These  computed  and  animation  results  graphically 
demonstrate  that  the  unsteady  flow  behavior  associated  with  the  wing  and  rotor  can  be  simulated. 


Figure  2  Iso-swirl  parameter  and  wing  surface  pressure  visualized  by  SimCenter’s 

visualization  software  DIVA 


2.4  User  Community 

SimCenter’s  unstructured  simulation  system  has  been  distributed  to  several  governmental 
agencies  and  industries  for  computational  simulation  and  design  analysis.  Major  users  of  the 
unstructured  grid  generation  include:  Boeing,  Rocketdyne,  Lockheed-Martin,  DIA-MSIC, 
NASA-Marshall,  PET  program,  in  addition  to  many  DoD  centers  that  use  the  software,  such  as 
NSWC-Carderock,  NSWC-Patuxent  River,  etc. 

The  U2NCLE  solver  has  been  transferred/distributed  to  NSWC-Carderock,  DIA-MISC, 
and  ARL/VTD.  The  U2NCLE  solver  is  currently  being  used  to  conduct  CFD  analysis  of  QUAD 
aero  interactions  for  Bell  Helicopter  Textron,  Inc.  in  support  of  Army’s  Joint  Heavy  Lift 
Rotorcraft  efforts.  The  U2NCLE  simulation  package  will  be  transferred  to  Bell  Helicopter  under 
a  contract  agreement  by  the  end  of  this  year. 
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In  additional  to  the  software  transfer,  techniques  and  algorithms  developed  at  the 
SimCenter/MSU  have  been  transferred  to  the  Air  Force  Research  Laboratory/ Air  vehicles 
Directorate  to  enhance  the  low  Mach  number  capability  for  the  Air  Vehicles  Unstructured  Solver 
(AVUS). 

The  following  is  a  list  of  key  users  of  the  U  NCLE  solver: 

•  Dr.  Joe  Gorski 

Naval  Surface  Warfare  Center-Carderock  Division,  MD 
Phone:  301-227-1930,  Email:  gorskijj@nswccd.navy.mil 

•  Dr.  Alan  Nicholson 

Missile  and  Space  Intelligence  Center,  Huntsville,  AL 
Phone:  256-313-7418,  Email:  lan@msic.dia.mil 

•  Dr.  Michael  Hathaway 

U.S.  Army  Research  Laboratory/ Vehicle  Technology  Directorate 
Phone:  216-433-6250,  Email:  Michael.D.Hathaway@grc.nasa.gov 

•  Jim  Narramore 

Bell  Helicopter  Textron,  Inc.,  Fort  Worth,  TX 

Phone:  817-280-5067,  Email:  jnarramore@bellhelicopter.textron.com 


3.  Computed  Results 


3.1  Grid  Resolution  Study 

A  series  of  mixed-element  unstructured  meshes  with  various  surface  resolutions  has  been 
generated  with  the  unstructured  grid  generation  system  SolidMesh  and  AFLR.  Grid  refinement 
study  has  been  conducted  for  the  NACA0015  wing  at  12  degrees  angle  of  attack  using  U2NCLE 
unstructured  flow  solver.  Computed  forces  and  moment  over  three  NACA0012  meshes  are  given 
in  Table  1,  which  shows  only  slight  differences  in  calculated  integrated  surface  loads  among 
these  meshes.  Therefore,  only  solutions  obtained  from  the  second  grid  (3.66M  node  points)  were 
reported  here. 

Computed  lift,  drag,  and  pitching  moment  coefficients  reported  here  are  defined  as: 

Cl  =  Lift  /  A. oqx;  Cd=  Drag  /  Apqx ;  Cm  =  Moment  /  cApqx 


where  qfj  =^p\f  is  the  dynamic  pressure,  and  Ap  is  the  wing  planform  area  estimated  as 

Ap  =  cxs ,  c  is  the  airfoil  chord,  and  s  is  the  airfoil  half-span  since  only  half  a  wing  physically 
mounted  on  the  wall  for  the  test  problem. 


Grids  and 

Turbulence  Models 

Ci =L  ift/ (Ap  *q  f 

Cd=Drag/(Ap  *qx) 

Cm  =Moment/( c  *AP  *qoo) 

Grid  1 
1.57M 

SA 

- 

- 

- 

kco 

0.886 

0.063 

0.012 

Grid  2 
3.66M 

SA 

0.901 

0.061 

0.010 

kco 

0.891 

0.063 

0.011 

Grid  3 
7.00M 

SA 

0.878 

0.064 

0.0094 

kco 

- 

- 

- 

Table  1  -  Computed  integrated  surface  loads  over  three  unstructured  meshes 
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3.2  Steady  State  Simulations 

Steady  state  simulations  for  the  NACA0015  wing  were  carried  out  in  parallel  using  64 
processors  on  a  Linux  cluster.  The  wing  was  positioned  at  12  degrees  angle  of  attack,  with  a  free 
stream  (reference)  Mach  number  of  0.1235  and  a  Reynolds  number  of  1.5  million  based  on  the 
airfoil  chord  length.  One-equation  Spalart-Allmaras  and  two-equation  k-oo  turbulence  models 
have  been  used  in  the  current  simulations.  For  a  steady  state  solution,  a  local  time  steeping  is 
used,  with  a  maximum  CFL  number  of  40  for  Spalart-Allmaras  model  and  10  for  k-oo  turbulence 
model.  Since  the  steady  state  simulation  does  not  require  temporal  accuracy,  only  one  Newton 
iteration  is  used  at  each  time  step,  with  10  symmetric  Gauss-Seidel  relaxations  at  each  Newton 
step.  The  current  U2NCLE  solver  has  a  low  Mach  number  capability  designed  for  low-speed 
flow  calculations  using  a  preconditioning  technique.  This  capability  can  be  easily  turned  on  or 
off  through  a  control  parameter  in  the  U2NCLE  input  file.  Numerical  tests  have  been  performed 
for  the  NACA0015  wing  with  and  without  preconditioning,  and  convergence  and  accuracy  of 
these  solvers  have  been  evaluated.  Figure  3  shows  the  convergence  histograms  of 
preconditioned  and  non-preconditioned  solutions  with  Spalart-Allmaras  and  k-cd  turbulence 
models.  With  the  preconditioning,  solutions  of  both  turbulence  models  show  further  reduction  in 
residual  comparing  to  the  non-preconditioned  counterparts. 

Computed  lift,  drag,  and  pitching  moment  coefficients  are  shown  in  Figures  4-6. 
Computed  forces/moment  by  the  preconditioned  scheme  reached  to  a  steady  state  much  quickly 
than  the  ones  without  using  the  preconditioning,  indicating  a  significant  savings  in  computing 
time  for  low  Mach  number  flows.  Furthermore,  differences  in  integrated  surface  loads  on  the 
wing  were  observed  between  preconditioned  and  non-preconditioned  solutions,  due  to  the 
difference  in  numerical  flux  formulations  between  the  two  schemes.  Generally  speaking,  a 
preconditioned  solution  offers  a  better  accuracy  and  convergence  than  a  non-preconditioned 
solution  at  low  Mach  numbers. 


Figure  3  -  Convergence  histories  with  and  without  preconditioning 
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Figure  4  -  Computed  lift  coefficients  with  and  without  preconditioning 


Figure  5  -  Computed  drag  coefficients  with  and  without  preconditioning 
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Time  Step 


Figure  6  -  Computed  moment  coefficients  with  and  without  preconditioning 

3.4  Time-Accurate  Simulations 

The  unsteady  time-accurate  simulations  were  performed  with  a  second-order  temporal 
accuracy  using  a  minimum  time  step  of  0.002,  which  is  equivalent  to  500  time  steps  per 
characteristic  time.  The  characteristic  time  T  is  the  non-dimensional  time  required  for  a  particle 
moving  at  the  free  stream  velocity  to  travel  one  chord  length.  Simulations  were  carried  out  on  a 
Linux  cluster  using  64  processors.  The  numerical  tests  have  been  conducted  to  evaluate  the 
effect  of  multiple  Newton  iterations  for  unsteady  runs.  Figure  7  shows  convergence  histories  of 
time-accurate  simulations  using  1,  2,  and  4  Newton  iterations.  The  results  show  that  multiple 
Newton  iterations  at  each  time  step  significantly  speed  up  the  solution  process  to  the  final  state 
for  the  test  problem.  However,  the  cost  of  CPU  time  is  also  increased  accordingly.  Time 
histories  of  integrated  lift,  drag,  and  pitch  moment  coefficients  on  the  wing  are  plotted  in  Figures 
8-10,  where  a  k-oo  turbulence  model  was  used.  By  comparing  the  relative  errors  of  computed 
unsteady  surface  loads  at  a  characteristic  time  of  15T  with  the  final  converged  (steady  state) 
solutions,  it  seems  that  the  time-accurate  solution  with  2  Newton  iterations  per  time  step  is  the 
most  cost-effective  way  to  reach  a  steady  state  for  the  current  test  problem.  However,  it  is 
recommended  that  3  to  4  Newton  iterations  be  used  at  each  time  step  to  achieve  the  temporal 
accuracy  for  complex  unsteady  simulations.  Figures  11-20  show  Cp  distributions  at  different 
wing  span  locations  obtained  with  4  Newton  iterations  at  a  characteristic  time  of  15T.  Computed 
results  from  the  converged  steady  state  solution  were  also  shown  in  the  figures  for  comparison 
purpose.  A  preconditioning  scheme  is  used  for  all  of  the  time-accurate  unsteady  solutions  shown 
here. 
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Figure  7  -  Convergence  of  time-accurate  solutions  with  multiple  Newton  iterations 

and  k-oo  turbulence  model 


Figure  8  -Time  history  of  integrated  lift  coefficients 
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Figure  9  -  Time  history  of  integrated  drag  coefficients 


Figure  10  -  Time  history  of  integrated  pitching  moment  coefficients 
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Figure  12  -  Computed  Cp  distributions  at  20%  span 
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Figure  13  -Computed  Cp  distributions  at  30%  span 
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Figure  14  -  Computed  Cp  distributions  at  40%  span 
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Figure  15  -  Computed  Cp  distributions  at  50%  span 
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Figure  16  -  Computed  Cp  distributions  at  60%  span 
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Figure  17-  Computed  Cp  distributions  at  70%  span 
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Figure  18  -  Computed  Cp  distributions  at  80%  span 
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Figure  19  -  Computed  Cp  distributions  at  90%  span 
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Figure  20  -  Computed  Cp  distributions  at  95%  span 
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3.5  Performance  and  Statistics 

The  scalability  of  this  flow  solver  for  a  Linux  supercluster  with  32,  64,  128,  and  256 
processors  is  shown  in  Figure  21  for  the  test  problem  with  3.66M  points.  Memory  and  CPU  time 
usages  are  listed  in  Table  2.  The  unstructured  solver  is  MPI  portable  across  a  wide  of  computer 
platforms,  and  the  present  results  have  been  obtained  on  a  874-processor  Linux  Supercluster 
(IBM  x330  1 .266  GHz  Pentium  III,  1 .3  GB  RAM  per  node). 

For  the  steady  simulation  using  64  processors,  the  total  CPU  time  required  for  a  converged 
solution  using  a  Spalart-Allmras  model  is  357  hours  (sum  of  CPU  time  on  64  processors  for  600 
time  steps).  For  the  unsteady  time-accurate  simulation  using  4  Newton  iterations  andK-oo 
turbulence  model,  the  total  CPU  time  required  to  run  15T  is  4467  hours  (sum  of  CPU  time  on  64 
processors  for  7500  time  steps).  All  CPU  times  were  counted  based  on  the  Linux  cluster 
mentioned  above. 

The  parallel  algorithms  for  unstructured  and  structured  grids  are  closely  related  and  have 
similar  scalability  properties.  The  scalability  properties  of  the  structured-grid  algorithm  have 
been  studied  for  generic  architectures  in  [6].  A  semi-empirical  performance  model  was 
developed  to  study  the  scalability  of  the  parallel  solution  algorithms  as  actually  implemented  for 
existing  and  9  hypothetical  computing  platforms  using  MPI  message  passing.  The  model  defines 
and  estimates  three  parallel  efficiencies  (CPU,  cost  and  memory  efficiencies),  and  results  are 
shown  in  Figure  22  for  a  generic  computer  having  the  following  parameters:  effective  CPU 
(lOOMflops)  and  buffering  (30Mb/s)  rates,  MPI  bandwidth  (130Mb/s)  and  latency  (15ms),  and 
memory  of  512Mb  per  processor.  These  are  shown  for  both  memory-constrained  size-up  in 
which  the  problem  size  is  increased  to  maintain  100%  memory  utilization  as  processors  are 
added,  and  a  constant-problem-size  scale-up  for  10  million  grid  points.  As  expected,  the  CPU 
efficiency  is  higher  for  memory-constrained  size-up,  but  the  difference  in  cost  efficiency  is  more 
dramatic  due  to  the  rapid  drop  in  memory  efficiency  for  constant  problem  size. 


Figure  21  -  Relative  efficiency  and  speedup  of  U2NCLE  solver  on  a  Linux  cluster 

for  the  test  problem  with  3.66M  node  points 
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Number  of 
Processor 

Memory  per 
processor  (MB) 

CPU  Time  per  Step 
(Seconds) 

Total  CPU  Time* 
(Hours) 

32(N) 

549 

175.5 

4.87 

64(2N) 

287 

114.6 

3.18 

128(4N) 

145 

60.4 

1.67 

256(8N) 

85 

30.3 

0.84 

Table  2  -  Memory  and  CPU  time  usages  per  processor  on  a  Linux  cluster 

for  the  test  problem  with  3.66M  node  points 

*Total  CPU  time  per  processor  required  to  run  100  steps  in  time-accurate  mode 
4  Newton  iterations  used  at  each  time  step 


Figure  22  -  Scalability  properties  from  a  semi-empirical  performance  model 

The  most  important  computer  parameters  for  scalability  are  effective  CPU  rate  as  timed 
for  the  executable  operating  with  message  passing  suppressed,  the  MPI  software  bandwidth  for 
large  messages,  and  the  time  required  to  load  message-passing  buffer  arrays.  The  MPI  latency  is 
negligible  since  there  are  only  a  small  number  of  large  messages.  Overall,  the  performance 
model  indicates  that  the  method  is  scalable  in  a  practical  sense  for  large-scale  problems. 
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4.  Optional  Results 


4.1  Overview 

The  optional  test  problem  considered  in  the  current  grant  is  a  realistic  three-bladed  rotor. 
It  has  a  radius  of  13  ft.  The  tip  speed  is  775.0  ft/s  and  the  tip  Mach  number  is  0.69.  The  Reynolds 
number  is  99  million  based  on  the  diameter  of  the  blade  and  the  tip  velocity.  The  reason  of 
choosing  this  configuration  is  to  demonstrate  the  unique  capability  of  the  U2NCLE  solver  to 
simulate  realistic  rotor  blade  motion  including  cyclic  pitching.  A  high-frequency  blade-pitch 
oscillation  is  introduced  to  the  hovering  and  descending  rotor.  Each  blade  is  oscillating  around 
its  own  axis  at  a  pitch  rate  of  4  cycles  per  rotor  revolution  and  at  an  amplitude  of  5  degrees.  This 
simulation  intends  to  mimic  a  Higher  Harmonic  Control  (HHC)  experiment  conducted  at  NASA 
Ames  Research  Center  for  reducing  blade-vortex  interaction  (BYI)  noise  for  XV- 15  Rotor  [12]. 


4.2  Unstructured  Grid 

The  computational  mesh  for  the  isolated  rotor  was  generated  with  the  SimCenter’s 
unstructured  grid  generation  tools  SolidMesh  and  AFLR.  The  mesh  has  2.29  million  nodes  and 
7.97  million  mixed  elements.  The  mesh  points  are  packed  near  the  solid  surface,  resulting  in 
boundary  layer  mesh  with  y+  value  of  about  one  off  the  solid  surface.  An  overall  view  of  the 
isolated  rotor  mesh  and  a  close  view  of  the  boundary  layer  mesh  near  the  blade  tip  are  shown  in 
Figure  23. 


Figure  23  An  overview  and  a  close-up  view  of  the  unstructured  mesh 
for  the  optional  test  problem 


4.3  Rotor  in  Hover 

First,  the  rotor  in  hover  is  modeled  with  the  U2NCLE  solver.  A  high-frequency  blade- 
pitch  oscillation  is  imposed  on  each  individual  blade  for  cyclic  motion.  The  instantaneous  blade 
pitch  angle  is  given  by  a(t)  =  ao  +  ai  sin(oot),  where  ao  is  the  collective  angle  and  ai  is  the 
alternating  pitch  angle  (amplitude  of  the  oscillation).  In  the  current  configuration,  the  collective 
angle  is  preset  and  alternating  pitch  angle  is  chosen  at  5  degrees.  The  input  pitch  rate  is  4  cycles 
per  blade  revolution.  A  time-accurate  viscous  flow  simulation  was  performed  using  4  Newton 
iterations  and  10  Gauss-Seidel  relaxations  at  each  time  step,  which  is  required  to  maintain  the 
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temporal  accuracy  and  stability  for  the  current  simulation.  The  minimum  time  step  used  is 
0.000292954  seconds  throughout  the  simulation,  which  corresponds  to  one  degree  of  the  rotor 
rotation  per  time  step.  A  periodical  solution  is  established  around  2000  time  steps,  or  about  5 
rotor  revolutions.  This  is  equivalent  to  28  hours  of  CPU  time  based  on  the  Linux  cluster 
mentioned  before. 

A  strong  tip-vortex  generated  by  the  rotor  blade  is  the  main  feature  of  the  unsteady  flow  field, 
where  the  trajectory  of  the  tip-vortex  can  last  several  revolutions  in  the  downwash  flow.  Figures  24 
and  25  show  the  instantaneous  swirl  parameter  and  Mach  number  on  the  rotor  disk  cutting  plane. 
Unsteady  vortex  patterns  of  the  tip-vortex  flow  are  observed  at  different  phases  during  the  blade 
upstroke  and  downstroke  oscillating  cycle.  The  complex  flow  behavior  associated  with  the  blade 
pitch  oscillation  is  clearly  demonstrated  by  the  current  unstructured  solver. 


Figure  24  -  Computed  swirl  parameter  on  rotor  disk  cutting  plane  during  downstroke  of 

blade  oscillating  cycle 


Figure  25  -  Computed  Mach  number  on  rotor  disk  cutting  plane  during 
downstroke  of  blade  oscillating  cycle 
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4.4  Rotor  in  Decent 

The  rotor  in  descent  condition  is  also  simulated  and  presented.  The  purpose  here  is  to 
simulate  and  demonstrate  the  complex  three-dimensional  unsteady  blade-vortex  interaction  (BVI) 
in  a  forward  and  descent  (approaching)  condition,  and  reveal  the  effect  of  active  blade-pitch 
control  to  the  flow  field.  The  simulation  was  performed  in  a  similar  manner  as  the  previous 
simulation,  except  that  the  free  stream  flow  has  an  advance  ratio  (ratio  of  the  forward  speed  to 
blade  tip  speed)  is  0.15,  and  rotor  descent  angle  is  8  degrees.  The  blade  pitch  rate  and  angle 
remain  the  same  as  the  previous  case.  The  flow  condition  is  considered  a  low  speed  forward- 
descent  flight,  where  the  blade-vortex  interaction  is  dominated  in  the  flow  field. 

For  rotor  without  blade-pitch  oscillation,  a  strong  tip  vortex  induced  by  the  advancing 
blade  is  clearly  seen  in  the  flow  field,  as  shown  in  Figure  26  by  the  swirl  parameter  on  the  rotor- 
disk  cutting  plane.  With  the  blade-pitch  oscillation,  the  trajectory  of  the  tip  vortex  generated  by 
the  advancing  blade  is  significantly  different  from  that  without  blade-pitch  oscillation,  as  shown 
in  Figure  27.  This  certainly  affects  the  way  how  the  blade  interacts  with  the  vortex,  and  thus  the 
BVI  noise.  The  computational  capability  demonstrated  in  the  current  unstructured  solver  may 
provide  better  understanding  to  this  complex  flow  phenomenon  and  underlying  mechanism  to  the 
BVI  noise  reduction  by  active  blade-pitch  control. 


Figure  26-  Computed  swirl  parameter  on  rotor  disk  plane  without  blade  oscillation 


Figure  27  -  Computed  swirl  parameter  on  rotor  disk  plane  with  blade  oscillation 
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5.  Technology  Transfer 


5.1  Solution  Transfer 

Unstructured  grid  and  solutions  produced  for  the  common  test  problem  NACA0015  wing 
have  been  transferred  to  Dr.  Robert  Meakin  at  Army  Aeroflightdynamics  Directorate  at  NASA 
Ames  Research  Center.  This  includes  an  unstructured  volume  grid  with  3.66M  node  points,  a 
time-accurate  solution  and  surface  Cp  distributions  at  a  specified  characteristic  time  15T 
obtained  by  the  U  NCLE  solver  with  k-co  turbulence  mode,  the  convergence  history  of  the 
forces/moment,  the  solver  input  and  boundary  condition  files  to  run  the  unstructured  solver 
U2NCLE. 

5.2  Software  Transfer 

Unstructured  simulation  package  was  successfully  transferred  to  Dr.  Robert  Meakin  at 
Army  Aeroflightdynamics  Directorate  at  NASA  Ames  Research  Center  in  April  2006.  Three 
versions  of  the  source  code  (incompressible,  compressible,  and  arbitrary  Mach  number)  for 
U2NCLE  were  released.  To  supplement  the  source  code,  an  instruction  note  and  a  web-based 
tutorial  describing  the  general  features  of  U  NCLE  were  delivered  in  the  package  to  Dr.  Bob 
Meakin  (AFDD).  A  password-protected  ftp  site  at  SimCenter/MSU  has  been  set  up  for 
downloading  the  following  simulation  packages  upon  request: 

•  SolidMesh  (SimCenter’s  surface  grid  generator) 

•  AFLR  (SimCenter’s  volume  grid  generator) 

•  DIVA  (SimCenter’s  visualization/animation  software) 

•  U  NCLE  (SimCenter’s  unstructured  flow  solver,  including  incompressible,  compressible, 
and  arbitrary  Mach  number  versions) 

•  USS_U2NCLE  (SimCenter’s  user  interface  software  for  grid  generation,  flow  solver,  and 
visualization) 

•  Miscellaneous  Unstructured  Grid  Tools  (checkgrid,  extract,  gc,  gridmerge,  part,  pcut, 
ptrace,  relframe,  scalegrid,  scaleflow,  slice,  surftool,  ugc,  ugio2cgns,  ugio2u2ncle,  unpart, 
unpartsurf,  voltool,  vortex) 


6.  Summary 

Steady  and  unsteady  time-accurate  simulations  have  been  performed  with  the  SimCenter’s 
unstructured  solver  U2NCLE  for  the  common  test  problem,  the  NACA0015  wing.  Results  and 
computational  statistics  from  these  simulations  have  been  obtained  and  presented  in  the  current 
report.  In  addition,  unsteady  time-accurate  simulations  have  been  performed  on  an  optional  test 
problem  about  a  generic  but  realistic  three-bladed  rotor  undergoing  high  frequency  blade-pitch 
motion.  Among  many  useful  features  in  the  U2NCLE  simulation  system,  the  following  remarks 
are  worth  to  mention  which  might  be  of  interest  to  the  HI-ARMS  evaluation  team: 

•  Performance  and  computational  statistics  indicate  that  the  U2NCLE  solver  is  scalable  in  a 
practical  sense  for  large-scale  problems. 
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•  The  arbitrary  Mach  number  version  of  the  U2NCLE  solver  allows  calculation  of  both 
low-  and  high-speed  viscous  flows,  with  demonstrated  benefit  in  convergence/accuracy 
in  predicting  low  Mach  number  flows. 

•  Dynamic  motion  grid  capability  in  the  U2NCLE  solver  allows  simulation  of  complete 
rotorcraft  with  actual  rotating  blades  undergoing  cyclic  blade  motion  in  a  fully 
conservative  way. 
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